Assignment #1

library(tidyverse)         # for graphing and data cleaning

## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.0 ──

## ✓ ggplot2 3.3.2     ✓ purrr   0.3.4
## ✓ tibble  3.1.0     ✓ dplyr   1.0.5
## ✓ tidyr   1.1.2     ✓ stringr 1.4.0
## ✓ readr   1.3.1     ✓ forcats 0.5.0

## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## x dplyr::filter() masks stats::filter()
## x dplyr::lag()    masks stats::lag()

library(tidymodels)        # for modeling

## ── Attaching packages ────────────────────────────────────── tidymodels 0.1.2 ──

## ✓ broom     0.7.3      ✓ recipes   0.1.15
## ✓ dials     0.0.9      ✓ rsample   0.0.9 
## ✓ infer     0.5.4      ✓ tune      0.1.3 
## ✓ modeldata 0.1.0      ✓ workflows 0.2.2 
## ✓ parsnip   0.1.5      ✓ yardstick 0.0.7

## ── Conflicts ───────────────────────────────────────── tidymodels_conflicts() ──
## x scales::discard() masks purrr::discard()
## x dplyr::filter()   masks stats::filter()
## x recipes::fixed()  masks stringr::fixed()
## x dplyr::lag()      masks stats::lag()
## x yardstick::spec() masks readr::spec()
## x recipes::step()   masks stats::step()

library(naniar)            # for analyzing missing values
library(vip)               # for variable importance plots

## 
## Attaching package: 'vip'

## The following object is masked from 'package:utils':
## 
##     vi

theme_set(theme_minimal()) # Lisa's favorite theme

hotels <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-02-11/hotels.csv')

## Parsed with column specification:
## cols(
##   .default = col_double(),
##   hotel = col_character(),
##   arrival_date_month = col_character(),
##   meal = col_character(),
##   country = col_character(),
##   market_segment = col_character(),
##   distribution_channel = col_character(),
##   reserved_room_type = col_character(),
##   assigned_room_type = col_character(),
##   deposit_type = col_character(),
##   agent = col_character(),
##   company = col_character(),
##   customer_type = col_character(),
##   reservation_status = col_character(),
##   reservation_status_date = col_date(format = "")
## )

## See spec(...) for full column specifications.

When you finish the assignment, remove the # from the options chunk at the top, so that messages and warnings aren’t printed. If you are getting errors in your code, add error = TRUE so that the file knits. I would recommend not removing the # until you are completely finished.

Setting up Git and GitHub in RStudio

Read the Quick Intro section of the Using git and GitHub in R Studio set of Course Materials. Set up Git and GitHub and create a GitHub repo and associated R Project (done for you when you clone the repo) for this homework assignment. Put this file into the project. You should always open the R Project (.Rproj) file when you work with any of the files in the project.

Task: Below, post a link to your GitHub repository.

https://github.com/neelloc28/Advanced-datasci-hw1.git

Creating a website

You’ll be using RStudio to create a personal website to showcase your work from this class! Start by watching the Sharing on Short Notice webinar by Alison Hill and Desirée De Leon of RStudio. This should help you choose the type of website you’d like to create.

Once you’ve chosen that, you might want to look through some of the other Building a website resources I posted on the resources page of our course website. I highly recommend making a nice landing page where you give a brief introduction of yourself.

Tasks:

• Include a link to your website below. (If anyone does not want to post a website publicly, please talk to me and we will find a different solution).

My website

• Listen to at least the first 20 minutes of “Building a Career in Data Science, Chapter 4: Building a Portfolio”. Go to the main podcast website and navigate to a podcast provider that works for you to find that specific episode. Write 2-3 sentences reflecting on what they discussed and why creating a website might be helpful for you.

Jacqueline and Emily mentioned how creating a portfolio (website) can serve multiple purposes, from motivating you to gain new skills, to getting you involved in the data science community, to helping you have something to show at a job interview. I felt inspired to potentially pursue personal projects in the future when they suggested that you think of a question that genuinely interests you and take it from there (even if it doesn’t specifically have to do with the industry you are in/hope to be in), because you can apply those skills you learn in other contexts. I think that creating a website will be helpful for me so that I can have a consolidated space that holds my projects, which will hopefully inspire me to continuously add to it over time.

• (Optional) Create an R package with your own customized gpplot2 theme! Write a post on your website about why you made the choices you did for the theme. See the Building an R package and Custom ggplot2 themes resources.

I ran into some errors that I couldn’t figure out when trying to make my package, so I will just leave the theme here for now and try to troubleshoot adding it to an R package later.

theme_beachy <- function () {
  theme_classic(base_size = 11, base_family = "Avenir") %+replace%
    theme(panel.background = element_blank(), #removes white background
          plot.background = element_rect(fill = "lightcyan1", color = NA), #change background to a pale blue
          axis.line = element_line(size = 0.3), #thinning out axis lines
          legend.position = c(0.9,0.95), #puts legend inside of plot
          legend.box.background = element_rect(fill="transparent", colour= NA), #make legend box background
          legend.box.margin = margin(6,6,6,6), #margin around legend
          legend.justification = c("right","top"),
          legend.box.just = "right",
          legend.background = element_rect(linetype = 1, size = 0.3, color = "black", fill = "lightyellow")
          
            )
}

#example of the theme used on the hotels dataset

ggplot(hotels, aes(x = meal, fill = as.factor(is_canceled))) +
  geom_bar(position = "stack") +
  ggtitle("Cancellations by meal type")+
  theme_beachy() +
  scale_fill_discrete(name = "Cancellation Status", breaks = c(0,1), labels = c("Didn't Cancel", "Canceled"))

Machine Learning review and intro to tidymodels

Read through and follow along with the Machine Learning review with an intro to the tidymodels package posted on the Course Materials page.

Tasks:

1. Read about the hotel booking data, hotels, on the Tidy Tuesday page it came from. There is also a link to an article from the original authors. The outcome we will be predicting is called is_canceled.

• Without doing any analysis, what are some variables you think might be predictive and why?

I think that lead_time (days elapsed between entering the date of the booking into the PMS and the arrival date) might be predictive because the reservations that are booked farther out ahead of time may be more likely to be canceled (due to unanticipated things coming up closer to the date of stay that would cause it to be canceled). I think previous_cancellations could be a predictor as well, because that may be a proxy for how reliable that customer is with committing to their booking. I also think that children (number of children) could be a predictor because if there are no children (and if it’s only one adult), then it may be more likely a business trip which isn’t as flexible as, say, a family vacation.

_ What are some problems that might exist with the data? You might think about how it was collected and who did the collecting.

Since customers often change the number of people, length of stay, and preferred room type once they arrive to the hotel, the data may not capture the true distribution of these variables (since the dataset only represents a snapshot in time). It also may be problematic that the data was only collected during summer months, when people are more likely to be on vacation (which could change their probability of canceling versus if it was for work).

• If we construct a model, what type of conclusions will be able to draw from it?

If we construct a model, hopefully we will be able to tell what factors (about the person booking, as well as the hotel) may make the booking more likely to be canceled. I can see how this could be problematic if the model is put into use because it could potentially result in discrimination for who hotels allow to book (if the hotel deems them likely to cancel).

2. Create some exploratory plots or table summaries of the variables in the dataset. Be sure to also examine missing values or other interesting values.

#checking to see the distribution of canceled vs. not canceled reservations
ggplot(hotels, aes(x = is_canceled)) +
  geom_bar()

#binarizing previous cancellations
hotels <- hotels %>%
  mutate(previous_cancellations_bin = ifelse(previous_cancellations == 0, "no", ifelse(previous_cancellations > 0, "yes", NA)))

ggplot(hotels, aes(x = previous_cancellations_bin)) +
  geom_bar()

Distributions of the quantitative variables

hotels %>% 
  select(where(is.numeric)) %>% 
  pivot_longer(cols = everything(),
               names_to = "variable", 
               values_to = "value") %>% 
  ggplot(aes(x = value)) +
  geom_histogram() +
  facet_wrap(vars(variable), 
             scales = "free")

## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

## Warning: Removed 4 rows containing non-finite values (stat_bin).

What I noticed:

*One weird spike in arrival_date_day_of_month

*Most are very right-skewed or have the majority of values at 0 or 1 (ex. lead_time, booking_change, babies, adults)

Check the class of each variable:

lapply(hotels, class)

## $hotel
## [1] "character"
## 
## $is_canceled
## [1] "numeric"
## 
## $lead_time
## [1] "numeric"
## 
## $arrival_date_year
## [1] "numeric"
## 
## $arrival_date_month
## [1] "character"
## 
## $arrival_date_week_number
## [1] "numeric"
## 
## $arrival_date_day_of_month
## [1] "numeric"
## 
## $stays_in_weekend_nights
## [1] "numeric"
## 
## $stays_in_week_nights
## [1] "numeric"
## 
## $adults
## [1] "numeric"
## 
## $children
## [1] "numeric"
## 
## $babies
## [1] "numeric"
## 
## $meal
## [1] "character"
## 
## $country
## [1] "character"
## 
## $market_segment
## [1] "character"
## 
## $distribution_channel
## [1] "character"
## 
## $is_repeated_guest
## [1] "numeric"
## 
## $previous_cancellations
## [1] "numeric"
## 
## $previous_bookings_not_canceled
## [1] "numeric"
## 
## $reserved_room_type
## [1] "character"
## 
## $assigned_room_type
## [1] "character"
## 
## $booking_changes
## [1] "numeric"
## 
## $deposit_type
## [1] "character"
## 
## $agent
## [1] "character"
## 
## $company
## [1] "character"
## 
## $days_in_waiting_list
## [1] "numeric"
## 
## $customer_type
## [1] "character"
## 
## $adr
## [1] "numeric"
## 
## $required_car_parking_spaces
## [1] "numeric"
## 
## $total_of_special_requests
## [1] "numeric"
## 
## $reservation_status
## [1] "character"
## 
## $reservation_status_date
## [1] "Date"
## 
## $previous_cancellations_bin
## [1] "character"

None of the variables are categorical (or binary) at the moment, but ideally hotel, is_cancelled, stays variables, adults, children, meals, etc. should be.

Now, let’s examine the missing values in this dataset.

gg_miss_var(hotels)

#It looks as though children is the only variable with missing values, and there are only four missing values, so I don't think too many NAs is a problem in this dataset

#downsize dataset to a few variables we may care most about
hotels_small <- hotels %>%
  select(hotel, is_canceled, lead_time, children, previous_cancellations, adults)
  
vis_miss(hotels_small)

#looks like there are barely any missing values!

ggplot(hotels_small, aes(x = children, y = as.factor(is_canceled))) +
  geom_miss_point()

#So all of the missing values in the children variable are reservations that were cancelled...

3. First, we will do a couple things to get the data ready, including making the outcome a factor (needs to be that way for logistic regression), removing the year variable and some reservation status variables, and removing missing values (not NULLs but true missing values). Split the data into a training and test set, stratifying on the outcome variable, is_canceled. Since we have a lot of data, we’re going to split the data 50/50 between training and test. I have already set.seed() for you. Be sure to use hotels_mod in the splitting.

hotels_mod <- hotels %>% 
  mutate(is_canceled = as.factor(is_canceled)) %>% 
  mutate(across(where(is.character), as.factor)) %>% 
  select(-arrival_date_year,
         -reservation_status,
         -reservation_status_date) %>% 
  add_n_miss() %>% 
  filter(n_miss_all == 0) %>% 
  select(-n_miss_all)

set.seed(494)

Split data into training and testing (50/50)

#randomly assign 50% of the data to training
hotels_split <- initial_split(hotels_mod, prop = .5)

hotels_split

## <Analysis/Assess/Total>
## <59693/59693/119386>

#training and testing datasets
hotels_training <- training(hotels_split)
hotels_testing <- testing(hotels_split)

4. In this next step, we are going to do the pre-processing. Usually, I won’t tell you exactly what to do here, but for your first exercise, I’ll tell you the steps.

• Set up the recipe with is_canceled as the outcome and all other variables as predictors (HINT: ~.).
  
• Use a step_XXX() function or functions (I think there are other ways to do this, but I found step_mutate_at() easiest) to create some indicator variables for the following variables:children,babies, andprevious_cancellations`. So, the new variable should be a 1 if the original is more than 0 and 0 otherwise. Make sure you do this in a way that accounts for values that may be larger than any we see in the dataset.
  
• For the agent and company variables, make new indicator variables that are 1 if they have a value of NULL and 0 otherwise.
• Use fct_lump_n() to lump together countries that aren’t in the top 5 most occurring.
• If you used new names for some of the new variables you created, then remove any variables that are no longer needed.
• Use step_normalize() to center and scale all the non-categorical predictor variables. (Do this BEFORE creating dummy variables. When I tried to do it after, I ran into an error - I’m still investigating why.)
• Create dummy variables for all factors/categorical predictor variables (make sure you have -all_outcomes() in this part!!).
  
• Use the prep() and juice() functions to apply the steps to the training data just to check that everything went as planned.

hotels_recipe <- recipe(is_canceled ~ ., data = hotels_training) %>%
  step_mutate(children = ifelse(children > 0,1,0), babies = ifelse(babies > 0,1,0), previous_cancellations = ifelse(previous_cancellations > 0,1,0)) %>%
  step_mutate(agent_null = ifelse(agent == 'NULL', 1, 0), company_null = ifelse(company=='NULL',1,0)) %>%
  step_mutate(country = fct_lump_n(country,n=5)) %>%
  step_rm(agent, company) %>%
  step_normalize(all_predictors(),-all_nominal()) %>% #numeric variables will now have standard deviation of 1 and mean of zero
  step_dummy(all_nominal(),-all_outcomes())


#using prep and juice to make sure everything went as planned
hotels_recipe %>%
  prep(hotels_training) %>%
  juice()

5. In this step we will set up a LASSO model and workflow.

• In general, why would we want to use LASSO instead of regular logistic regression? (HINT: think about what happens to the coefficients).

We would want to use LASSO instead of regular logistic regression to better avoid overfitting our model to the training dataset (thereby having it do poorly with the testing and other datasets). LASSO assigns a penalty to less important coefficients to shrink them to zero. The penalty (\(\lambda\) can be from 0 to infinity) that minimizes the sum of the squared residuals is the one that LASSO chooses.

• Define the model type, set the engine, set the penalty argument to tune() as a placeholder, and set the mode.

hotels_lasso_mod <-
  # Define a LASSO model
  logistic_reg(mixture=1) %>%
  # Set the engine
  set_engine("glmnet") %>%
  #parameters we will tune
  set_args(penalty = tune()) %>%
  # Set mode
  set_mode("classification")

hotels_lasso_mod

## Logistic Regression Model Specification (classification)
## 
## Main Arguments:
##   penalty = tune()
##   mixture = 1
## 
## Computational engine: glmnet

• Create a workflow with the recipe and model. (combines the preprocessing and model definition steps)

hotels_lasso_wf <-
  #set up the workflow
  workflow() %>%
  #add the recipe
  add_recipe(hotels_recipe) %>%
  #add the modeling
  add_model(hotels_lasso_mod)

hotels_lasso_wf

## ══ Workflow ════════════════════════════════════════════════════════════════════
## Preprocessor: Recipe
## Model: logistic_reg()
## 
## ── Preprocessor ────────────────────────────────────────────────────────────────
## 6 Recipe Steps
## 
## ● step_mutate()
## ● step_mutate()
## ● step_mutate()
## ● step_rm()
## ● step_normalize()
## ● step_dummy()
## 
## ── Model ───────────────────────────────────────────────────────────────────────
## Logistic Regression Model Specification (classification)
## 
## Main Arguments:
##   penalty = tune()
##   mixture = 1
## 
## Computational engine: glmnet

6. In this step, we’ll tune the model and fit the model using the best tuning parameter to the entire training dataset.

• Create a 5-fold cross-validation sample. We’ll use this later. I have set the seed for you.
  
• Use the grid_regular() function to create a grid of 10 potential penalty parameters (we’re keeping this sort of small because the dataset is pretty large). Use that with the 5-fold cv data to tune the model.
  
• Use the tune_grid() function to fit the models with different tuning parameters to the different cross-validation sets.
  
• Use the collect_metrics() function to collect all the metrics from the previous step and create a plot with the accuracy on the y-axis and the penalty term on the x-axis. Put the x-axis on the log scale.
  
• Use the select_best() function to find the best tuning parameter, fit the model using that tuning parameter to the entire training set (HINT: finalize_workflow() and fit()), and display the model results using pull_workflow_fit() and tidy(). Are there some variables with coefficients of 0?

set.seed(494) # for reproducibility

#5-fold cross-validation
hotels_cv <- vfold_cv(hotels_training, v = 5)

#choose penalty parameter values
penalty_grid <- grid_regular(penalty(), levels = 10)
penalty_grid

#fit the models with different tuning parameters to the different cross-validation sets
hotels_lasso_tune <-
  hotels_lasso_wf %>%
  tune_grid(
    resamples = hotels_cv,
    grid = penalty_grid
  )

hotels_lasso_tune

hotels_accuracy <- hotels_lasso_tune %>%
  collect_metrics() %>%
  filter(.metric == "accuracy")

hotels_accuracy

ggplot(hotels_accuracy, aes(x = penalty, y = mean)) +
  geom_point() +
  geom_line() +
  scale_x_log10(
   breaks = scales::trans_breaks("log10", function(x) 10^x),
   labels = scales::trans_format("log10",scales::math_format(10^.x))) +
  ggtitle("Accuracy over different penalties") +
  ylab("accuracy")

#select best tuning parameter (accuracy-wise)
best_param_hotels <- hotels_lasso_tune %>%
  select_best(metric = "accuracy")

best_param_hotels

Fitting the model with the best tuning parameter according to select_best

#create workflow
hotels_lasso_final_wf <- hotels_lasso_wf %>%
  finalize_workflow(best_param_hotels)

hotels_lasso_final_wf

## ══ Workflow ════════════════════════════════════════════════════════════════════
## Preprocessor: Recipe
## Model: logistic_reg()
## 
## ── Preprocessor ────────────────────────────────────────────────────────────────
## 6 Recipe Steps
## 
## ● step_mutate()
## ● step_mutate()
## ● step_mutate()
## ● step_rm()
## ● step_normalize()
## ● step_dummy()
## 
## ── Model ───────────────────────────────────────────────────────────────────────
## Logistic Regression Model Specification (classification)
## 
## Main Arguments:
##   penalty = 1e-10
##   mixture = 1
## 
## Computational engine: glmnet

#fit to the training data
hotels_lasso_final_mod <- hotels_lasso_final_wf %>%
  fit(data = hotels_training)

hotels_lasso_final_mod %>% 
  pull_workflow_fit() %>% 
  tidy() 

Yes, some variables now have coefficients of zero: arrival_date_month_October, meal_FB, market_segment_Groups, market_segment_Undefined, distribution_channel_Undefined, assigned_room_type_L, and assigned_room_type_P.

7. Now that we have a model, let’s evaluate it a bit more. All we have looked at so far is the cross-validated accuracy from the previous step.

• Create a variable importance graph. Which variables show up as the most important? Are you surprised?

# Visualize variable importance
hotels_lasso_final_mod %>% 
  pull_workflow_fit() %>% 
  vip()

From the variable importance plot, we see that reserved_room_type_P, deposit_type_Non.Refund, and assigned_room_type_I are the top three most important predictors of canceling a reservation. I am not sure what room types P and I are, but I am not surprised that putting a non-refundable deposit down is a strong indicator for whether or not the booking got canceled (because people don’t like to lose money so they may be less likely to cancel). If room type indicated the size and number of beds, I could see it being a relevant predictor because people going on business trips (small bed, single room most likely) may be less likely to cancel.

• Use the last_fit() function to fit the final model and then apply it to the testing data. Report the metrics from the testing data using the collect_metrics() function. How do they compare to the cross-validated metrics?

# Fit model with best tuning parameter(s) to training data and apply to test data
hotels_lasso_test <- hotels_lasso_final_wf %>% 
  last_fit(hotels_split)

# Metrics for model applied to test data
hotels_lasso_test %>% 
  collect_metrics()

#metrics for model applied to cv data
hotels_lasso_tune %>%
  collect_metrics()

#comparing metrics
ggplot(hotels_accuracy, aes(x = penalty, y = mean)) +
  geom_point() +
  geom_line() +
  scale_x_log10(
   breaks = scales::trans_breaks("log10", function(x) 10^x),
   labels = scales::trans_format("log10",scales::math_format(10^.x))) +
  ggtitle("Accuracy over different penalties") +
  ylab("accuracy") +
  geom_point(aes(x=0.0004641589, y = 0.8128256, col = "red")) +
  labs(color = "Test data metric")

The accuracy of model applied to CV data with the best tuning parameter is SLIGHTLY lower than the accuracy of the model applied to the test data with the same tuning parameter. We can see this in the plot, where the red dot (test data) is slightly higher than the black dot (training data).

• Use the collect_predictions() function to find the predicted probabilities and classes for the test data. Save this to a new dataset called preds. Then, use the conf_mat() function from dials (part of tidymodels) to create a confusion matrix showing the predicted classes vs. the true classes. What is the true positive rate (sensitivity)? What is the true negative rate (specificity)? See this Wikipedia reference if you (like me) tend to forget these definitions.

# predictions from testing data
preds <- collect_predictions(hotels_lasso_test)
preds

#make a confusion matrix
conf_mat(preds, .pred_class, is_canceled)

##           Truth
## Prediction     0     1
##          0 34225  3340
##          1  7782 14346

• Use the preds dataset you just created to create a density plot of the predicted probabilities of canceling (the variable is called .pred_1), filling by is_canceled. Use an alpha = .5 and color = NA in the geom_density().

ggplot(preds, aes(x = .pred_1, fill = is_canceled)) +
  geom_density(alpha = 0.5, color = NA) +
  ggtitle("Predicted probabilities of cancelling (test data)")

Answer these questions:

1. What would this graph look like for a model with an accuracy that was close to 1?

A model with an accuracy that was close to 1 would have densities almost solely at x-axis values of 0 and 1.

2. Our predictions are classified as canceled if their predicted probability of canceling is greater than .5. If we wanted to have a high true positive rate, should we make the cutoff for predicted as canceled higher or lower than .5?

If we wanted a high true positive rate (correctly predicting canceled reservations as canceled) then we would want to make the cutoff for “predicted as canceled” lower than 0.5 because that would assign more predicted probabilities to canceled=yes.

3. What happens to the true negative rate if we try to get a higher true positive rate?

If we try to get a true positive rate, the true negative rate will subsequently be lower because we are classifying more reservations as “canceled” which thereby classifies less as “not canceled”.

8. Let’s say that this model is going to be applied to bookings 14 days in advance of their arrival at each hotel, and someone who works for the hotel will make a phone call to the person who made the booking. During this phone call, they will try to assure that the person will be keeping their reservation or that they will be canceling in which case they can do that now and still have time to fill the room. How should the hotel go about deciding who to call? How could they measure whether it was worth the effort to do the calling? Can you think of another way they might use the model?

Hotels could use this model to call anyone whom the model predicted would cancel their reservation, to confirm that they still wanted the reservation. They could measure whether pr not it was worth the effort to call by keeping track of the “true negatives” (people they called who actually were planning to cancel their reservation) and “false negatives” (people they called that the model predicted would cancel but weren’t actually planning to cancel) and seeing if the model guessed right more often than it guessed incorrectly (to a significant degree). Another way that this model could be used is to see who it predicts is NOT likely to cancel, and then sending them promotional material or catering to those people in some way to attempt to make them want to continue booking with that hotel company again.

9. How might you go about questioning and evaluating the model in terms of fairness? Are there any questions you would like to ask of the people who collected the data?

Depending on how the model would be used in the future, I might challenge their pick of the putting a non-refundable deposit down variable to predict cancellations, because some people are financially or logistically unable to do so. Therefore, if the model caused hotels to give them poorer treatment, that is unjust. I would also like to ask the people who collected the data why they chose to use data from hotels in Portugal, and how much they think the results would change if you compared across different countries.

Bias and Fairness

Listen to Dr. Rachel Thomas’s Bias and Fairness lecture. Write a brief paragraph reflecting on it. You might also be interested in reading the ProPublica article Dr. Thomas references about using a tool called COMPAS to predict recidivism. Some questions/ideas you might keep in mind:

• Did you hear anything that surprised you?
  
• Why is it important that we pay attention to bias and fairness when studying data science?
  
• Is there a type of bias Dr. Thomas discussed that was new to you? Can you think about places you have seen these types of biases?

Algorithmic bias is crucial to acknowledge because a biased algorithm can unjustly and permanently harm people peoples’ lives. Something I took away from this talk was the importance of examining accuracy on subgroups of the data, rather than just the accuracy of the algorithm as a whole. It is quite angering that machine learning algorithms that were proven to be harmfully biased are still allowed to be used. For example, even though the algorithm to predict recidivism had a high false positive rate for African American people and a high false negative rate for white people, Wisconsin still continued to use it. I don’t understand if it is just out of a desire for cost-effectiveness/ease/efficiency, lack of knowledge about the poor model performance, or something else. It just feels so cruel, racist (even though race wasn’t an input variable in that model), and unethical, and I am shocked that they were allowed to continue using it or that it even got put into use in the first place.

Another takeaway from the lecture is the need to gather more representative datasets to train your algorithm on, but also realizing that many of these classification variables are social constructs to begin with (race, gender…) and is impossible to be 100% sure about from just looking at a picture of someone.

Overall, it’s really unfortunate that people are more likely to assume algorithms are less biased (at least, according to the talk) than people making the decisions because people are creating those algorithms in the first place. I would pose that people have the ability to be more nuanced than algorithms when they make decisions because ideally they can take a step back and acknowledge their biases before making snap judgements (or pausing to challenge their snap judgements) about people that can make or break their futures.